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DETAILED ACTION 

This office action is responsive to the Request for Continued Examination (RCE) 
filed June 20, 2007. Claims 10, 11 and 15-18 are currently amended, claims 12-14 and 
19 are canceled, and claims 20-26 have been added. Therefore claims 10, 11, 15-18 
and 20-26 are pending in this application. 

Claim Rejections - 35 USC §112 

1. The following is a quotation of the second paragraph of 35 U.S.C. 112: 

The specification shall conclude with one or more claims particularly pointing out and distinctly 
claiming the subject matter which the applicant regards as his invention. 

2. Claims 17 and 18 are rejected under 35 U.S.C. 112, second paragraph, as being 
indefinite for failing to particularly point out and distinctly claim the subject matter which 
applicant regards as the invention. 

3. Claim 17 recites the limitation "the stored content" in line 3. There is insufficient 
antecedent basis for this limitation in the claim. 

4. Claim 18 recites the limitation "the stored content" in. lines 3-4. There is 
insufficient antecedent basis for this limitation in the claim. 

Claim Rejections - 35 USC § 103 

5. The following is a quotation of 35 U.S.C. 103(a) which forms the basis for all 
obviousness rejections set forth in this Office action: 

(a) A patent may not be obtained though the invention is not identically disclosed or described as set 
forth in section 1 02 of this title, if the differences between the subject matter sought to be patented and 
the prior art are such that the subject matter as a whole would have been obvious at the time the 
invention was made to a person having ordinary skill in the art to which said subject matter pertains. 
Patentability shall not be negatived by the manner in which the invention was made. 
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6. Claims 10, 15, 16, 18 and 20-22 are rejected under 35 U.S.C. 103(a) as being 
unpatentable over Harui (US Patent 6,690,394 B1) in view of Browne et al. (US 
2004/0135815 A1) ('Browne'). 

With respect to claim 10, Harui teaches a method for obtaining context 
information comprising the steps of: 

receiving a selection of an on-screen region of display, one or more graphical 
elements being displayed in the selected on-screen region (column 3 line 66 - column 4 
line 1); 

determining one ore more graphical elements associated with the selected on- 
screen region (column 4 lines 4-7); 

capturing data for displaying the one or more graphical elements (column 4 lines 
4-7), and storing the captured data (column 1 lines 59-63); 

obtaining context information for the one or more graphical elements from at 
least one of the following: 

identifying the date on which the image pixels are captured, 
identifying a user responsible for selecting the on-screen region, 
receiving a user annotation of the selected on-screen region (column 4 

lines 23-26), 

performing text recognition on the captured data (column 5 lines 63-67), 

and 
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querying an application causing the one or more graphical elements to be 
displayed when executed in the computer-based system; and 
storing the context information (column 8 lines 53-55). 

Harui does not explicitly teach image pixels or storing the captured image pixels 
in an image file, 

Browne teaches a method and apparatus for image metadata entry (see 
abstract), in which he teaches capturing image pixels for displaying the one or more 
.graphical elements, and storing the captured image pixels in an image file (paragraphs 
4-5, 11 and 173). 

It would have been obvious to a person having ordinary skill in the art at the time 
the invention was made to have modified Harui by the teaching of Browne because 
storing the captured image pixels in an image file would enable an easy and efficient 
method of classifying and storing digital images associated with the web pages of Harui 
(Browne, paragraph 14). 

With respect to claim 15, Harui as modified teaches creating and storing a linking 
structure as the association between the image file and the context information 
(Browne, Figure 12, paragraph 136). 
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With respect to claim 16, Harui as modified teaches storing an association 
between the stored context information and the image file (Harui, column 8 lines 53-55; 
Browne, paragraph 117). 

With respect to claim 18, Harui as modified teaches wherein the linking structure 
includes at least one pointer pointing to the stored image file or the stored content 
information (Browne, paragraph 136). 

With respect to claim 20, Harui as modified teaches wherein the context 
information is stored in such a manner as to be accessible to a user for performing at 
least one of the following: 

searching for said image file (Browne, paragraphs 61-65 and 125), 

displaying the context information simultaneously with the captured image pixels, 

and 

navigating a network to a source of the captured image pixels. 

With respect to claim 21 , Harui as modified teaches wherein the one or more 
graphical elements comprises a first set of one or more textual characters, the method 
further comprising: obtaining the context information as text data obtained by performing 
text recognition on at least one of: the first set of one or more textual characters, and a 
second set of textual characters displayed in proximity with the first set (Harui, column 5 
lines 54-67). 
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With respect to claim 22, Harui as modified teaches wherein the selected on- 
screen region is part of displayed textual region, and the graphical elements comprise a 
first set of one or more textual characters displayed in the textual region, the method 
further comprising: obtaining the context information based on a second set of one or 
more textual characters displayed in the textual region (Harui, column 5 lines 54-67). 

7. Claim 11, 25 and 26 are rejected under 35 U.S.C. 103(a) as being unpatentable 
over Harui (US Patent 6,690,394 B1) in view of Browne et al. (US 2004/0135815 A1) 
('Browne') as applied to claims 10, 15, 16, 18 and 20-22 above, and further in view of 
Oppermann et al. (US 6,334,157 B1) ('Oppermann'). 

With respect to claim 1 1 , Harui as modified teaches: 

determining a window associated with the selected on-screen region (column 4 
lines 4-7); and 

obtaining a URI property as context information (column 4 lines 23-26). 

Harui does not teach retrieving an application interface having a uniform 
resource identifier (URI) property from the determined window or parent window of the 
determined window. 

Oppermann teaches programmatically providing direct access to user interface 
elements of an application program (see abstract), in which he teaches: 
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retrieving an application interface having a uniform resource identifier (URI) 
property from the determined window or parent window of the determined window 
(column 25 lines 59-62, column 28 lines 33-39); and 

obtaining the URI property as the context information (column 1 1 lines 1-9, 55- 
60, column 12 lines 55-60). 

It would have been obvious to a person having ordinary skill in the art at the time 
the invention was made to have further modified Harui by the teaching of Oppermann 
because retrieving an application interface having a uniform resource identifier (URI) 
property from the determined window or parent window of the determined window would 
enable accessibility aids the ability to access and manipulate user interface elements of 
any application program without having prior knowledge of the application program or its 
interface (Oppermann, column 4 lines 27-30). 

With respect to claim 25, Harui as modified teaches wherein the selected on- 
screen region includes at least a portion of a displayed web page or document (Harui, 
column 3 lines 66-67), and the step d) further comprises: using an application 
programming interface (API) to query the application for the context information 
(Oppermann, column 7 lines 36-38). 

With respect to claim 26, Harui as modified teaches wherein the step d) further 
comprises obtaining a uniform resource identifier (URI) of the web page or document as 
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the context information (Harui, column 4 lines 23-26), the URI being obtained as a result 
of the query using the API (Oppermann, column 7 lines 36-38). 

8. Claim 17 is rejected under 35 U.S.C. 103(a) as being unpatentable over Harui 
(US Patent 6,690,394 B1) in view of Browne et al. (US 2004/0135815 A1) ('Browne') as 
applied to claims 10, 15, 16, 18 and 20-22 above, and further in view of Newman (US 
2003/0101156 A1). 

With respect to claim 17, Harui as modified teaches claim 15. 

Harui as modified does not teach wherein the linking structure is incorporated in 
a file separate from the stored image file and the stored content information. 

Newman teaches database systems and methods (see abstract), in which he 
teaches wherein the linking structure is incorporated in a file separate from the stored 
image file and the stored content information (paragraph 16). 

It would have been obvious to a person having ordinary skill in the art at the time 
the invention was made to have further modified Harui by the teaching of Newman 
because wherein the linking structure is incorporated in a file separate from the stored 
image file and the stored content information would enable additional information about 
image files, such as the origination device, person who created the file, and data/time 
the file was created, to be transmitted and stored along with the image files (Newman, 
paragraph 16). 
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9. Claims 23 and 24 are rejected under 35 U.S.C. 103(a) as being unpatentable 
over Harui (US Patent 6,690,394 B1) in view of Browne et al. (US 2004/0135815 A1) 
('Browne') as applied to claims 10, 15, 16, 18 and 20-22 above, and further in view of 
Holloway, JR (US 2005/0198354 A1). 

With respect to claim 23, Harui as modified teaches claim 10. 

Harui as modified does not teach wherein the step a) receives the selection 
based on movement of a stylus across the display. 

Holloway, JR teaches a job site communications system (see abstract), in which 
he teaches wherein the step a) receives the selection based on movement of a stylus 
across the display (paragraphs 57 and 58). 

It would have been obvious to a person having ordinary skill in the art at the time 
the invention was made to have further modified Harui by the teaching of Holloway, JR 
because wherein the step a) receives the selection based on movement of a stylus 
across the display would enable an easy method for inputting data and selecting data, 
and allow Harui's delivery system to be use for performing job functions (Holloway, JR, 
paragraph 55). 

With respect to claim 24, Harui as modified teaches: 

digitizing movements of a stylus across the display in order to receive the user 
annotation (Holloway, JR, paragraph 62); and 
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obtaining the context information based on the received user annotation 
(Holloway, JR, paragraphs 57 and 63). 

Response to Arguments 

10. Applicant's arguments with respect to claims 10, 11 and 15-18 have been 
considered but are moot in view of the new ground(s) of rejection. 

Conclusion 

Any inquiry concerning this communication or earlier communications from the 
examiner should be directed to Alicia M. Lewis whose telephone number is 571-272- 
5599. The examiner can normally be reached on Monday - Friday, 9 - 6:30, alternate 
Friday off. 

If attempts to reach the examiner by telephone are unsuccessful, the examiner's 
supervisor, Charles Rones can be reached on 571-272-4085. The fax phone number 
for the organization where this application or proceeding is assigned is 571-273-8300. 
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Information regarding the status of an application may be obtained from the 
Patent Application Information Retrieval (PAIR) system. Status information for 
published applications may be obtained from either Private PAIR or Public PAIR. 
Status information for unpublished applications is available through Private PAIR only. 
For more information about the PAIR system, see http://pair-direct.uspto.gov. Should 
you have questions on access to the Private PAIR system, contact the Electronic 
Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a 
USPTO Customer Service Representative or access to the automated information 
system, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000. 

Alicia Lewis 
August 28, 2007 
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